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Abstract 

We analyse the structure of imprecise Markov chains and study their 
convergence by means of accessibility relations. We first identify the sets of 
states, so-called minimal permanent classes, that are the minimal sets ca- 
pable of containing and preserving the whole probability mass of the chain. 
These classes generalise the essential classes known from the classical the- 
ory. We then define a class of extremal imprecise invariant distributions 
and show that they are uniquely determined by the values of the upper 
probability on minimal permanent classes. Moreover, we give conditions 
for unique convergence to these extremal invariant distributions. 

Keywords imprecise Markov chain, invariant expectation functional, ac- 
cessibility relation, convergence of imprecise Markov chains 
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1 Introduction 

The term imprecise Markov chain denotes a Markov chain whose parameters 
are not fully determined, but rather some partial information about them is 
given in terms of models of imprecise probabilities. The systematic modelling 
of Markov chains with par tially determined paramete r s was in i tiated by Hart- 



fiel and Seneta (see e.g. 



Hartfiel and Senetal . Il994 iHartfieJ Il998t and the 
references therein) under the name 'Markov set chains'. Their theory is not 
f ormally built on the models of imprecise probabilities, such as Walley's model 
( [Wallevl . ll99lj ). although they develop similar concepts and methods. More for- 
mally the models of imprecise probabilities h ave been inv olved in the study of 
Markov chains by the approaches pro posed by Skuli 12009 1. where general mod- 
els of interval probabilities (see e.g. Weichselbergeij . |2001| ) are involved, and 
de Cooman. Hermans, and Quaeghebeuii [2009j, who introduce the approach 
with upper and lower expectation functionals. 

One of the main considerations of all mentioned approaches are the condi- 
tions for unique convergence to an invariant (imprecise) limit distribution. It 
has been shown that under certain assumptions imprecise Markov chains be- 
have in a similar way as the precise ones, allowing the poss i bility to generalise 



concepts, suc h as acc e ssibil i ty relations de Cooman et al.l . l2009l |. coefficients 



of ergodicity HartfielL 1 19981 . 1 Skuli and Habld . |2012|, and, for the chains with 



certain absorption, the inv ariant distributions conditional on non-absorption 
Grossman and SkulilbOlOj . 



The main topic of the present paper are the imprecise Markov chains that 
do not converge uniquely, which means that the hmit imprecise distribution 
depends on the initial state or distribution over the set of states. In the classi- 
cal theory this case is relatively easily translated to the case with unique con- 
vergence by partitioning the set of states into disjoint communication classes, 
which are then either essential or transient. Essential classes have the property 
that the probability mass concentrated within them and in the transient classes 
leading to them is preserved within the class. In the long term it is then either 
distributed among states in a stationary way or moves between the states in 
cycles. Transient classes, though, in the long term transfer all the probabil- 
ity mass to the essential classes. The long term behaviour of a precise Markov 
chain is then described with the proportions of the probability mass within each 
essential class, that can then be examined, each separately without influencing 
the others, as e ither ergo die or cyclic single communication classes. (For more 
details see e.g. Seneta], |2006|.) 



The situation in the case of imprecise Markov chains is considerably more 
complex. In the precise case the probability mass can only move from one state 
into another, which is determined by strictly positive transition probability. In 
the imprecise case, though, it can happen that instead we have a probability 
interval that has a strictly positive upper and zero lower bound. Therefore, 
we have to take into account both scenarios - where a transition is possible 
and where it is not - within the same model (for better clarification see Ex- 
ample [T]). This leads to numerous interesting new possibilities that are not 
observed in the precise theory. One immediate consequence is that there are 
different possible ways to define communication relations between states. Say 
for instance, when the upper probability of transition from a state x to another 
state y is positive and the lower one zero, can we say that y is accessible from 
X or not? We argue that questions of this kind cannot be answered with a 
single acce ssibility relation. Theref ore, in addition to the accessibility relation 
defined by Ide Cooman et al.l 2009| , that is based on upper probabilities and 



which we describe in Subsection 13.11 we define another accessibility relation 
in Subsection 13.21 This relation is defined between sets of states rather than 
between single states. Combining both relations will allow us to define mini- 
mal classes of states, called minimal permanent classes, that generalise essential 
communication classes from the classical theory in the sense that they are the 
minimal sets of states capable of possibly preserving the entire probability mass 
concentrated within them. However, unlike in the case of essential classes in 
the classical theory, this property does not prevent the possibility that the mass 
that can possibly be preserved within one class is entirely or partially transferred 
elsewhere, that is to other minimal permanent classes. Moreover, these classes 
are neither necessarily disjoint. In effect, a complex network of interdependence 
between the classes in possible in general. Our first main result in Theorem [2] 
shows that nevertheless all invariant imprecise distributions that possibly con- 
centrate the entire probability mass in one or more minimal permanent classes 
are uniquely determined by the set of such classes. Further, our second main 
result in Theorem [3] shows that when probability mass is at least in the limit 
concentrated in these classes the imprecise distributions over the set of states 
converge to the unique invariant distributions. These two theorems present the 
main results of this paper. 

In addition to the main theorems, we believe that the analysis of the pair of 



accessibility relations contributes to better understanding of imprecise Markov 
chains in general, describing the complexity of their structure, and on the other 
hand, the induced minimal permanent classes seem to present the smallest irre- 
ducible units, counterparts to the essential communication classes in the classical 
Markov chains. Although, considerably more apparently irreducible complexity 
remains in the imprecise case due to complex interdependencies that may be 
present between the classes. 

The paper has the following structure. In the next section we formally 
describe the model of imprecise Markov chains. In Section |3] we first de- 
scr ibe the accessibili t y rela tion based on upper transitions, previously defined 
by Ide Cooman et al.l 2009J, which here is referred as the weak accessibility re- 



lation, and state the unique convergence theorem, that was one of the main 
results in the above reference. The so-called strong accessibility relation is then 
defined in Subsection l3.21 which allows the definition of the so-called permanent 
classes in Subsection 13.31 In Section U] we finally use these results to prove our 
main results in Theorem [2] and Theorem [3l 

2 Imprecise Markov chains 

Let A" be a non-empty finite set of states and {X„}„£n a sequence of random 
variables taking values in X. The probability distributions of the variables are 
assumed to be imprecise. Throughout the paper an imprecise probability (distri- 
bution) denotes a closed convex set of expectation functionalqj i5. The elements 
of this set will be denoted by P. Sets of expectation functionals are of course 
equivalent to sets of corresponding probability distributions. Instead of the 
term imprecise probability distribution, we will call E an imprecise expectation 
functional (lEFj^ 

In our case where the set of states is finite every lEF is a compact set. 
A convenient way to represent lEFs is via their lower and upper expectation 
functionals as follows. Let E be an lEF and / a real valued map A" — > M. Then 
the lower and the upper expectation with respect to E are 



^(/) = minP(/) 



and 



i?(/)=maxP(/). 

We will call real valued maps / : A" — >■ M gambles. The set of all gambles on X 
is denoted by C{X), or simply by £. The values of P{f) for a given gamble / 
and all functionals P ^ E therefore form the interval [:E(/), _E(/)], and we may 
therefore identify E{f) with this interval. In particular, when / = 1a, where A 
is some set of states, the value P{Ia) denotes the probability of A, and K{^a) 
and E[1a) are the lower and the upper probability of A. 



^The notation here is very simplified version of the notation used bv I Wallevl Il99lll . which 
is still enough general and clear for our purpose. 

■^Imprecise expectation functionals correspond to lower and upper previsions which con- 
stitute one of the key objects studied in the theory of imprecise probabilities. The way how 
imprecise expectation functionals are used here corresponds to credal sets of lower and upper 
previsions. 



The set of lEFs is ordered with respect to set inclusion, and in the case where 
E C F we say that E is more committal than F. Clearly, an lEF E is more 
committal than F if and only ii E_{f) > F{f), or equivalently, E{f) < F{f), for 
every gamble /. 

The subset of gambles mapping X to the interval [0, 1] will be denoted by 
jCi, and it is a compact set. It can be easily verified that the values of imprecise 
expectation functionals on Ci uniquely determine their values on the whole C. 
Particularly, E{f) < F{f) for every gamble f € Ci implies E C F. 

The least committal imprecise expectation functional on C{X) is the vacuous 
lEF which contains all expectation functionals. The vacuous lEF will usually 
be denoted by V. For every gamble / 6 J0{X) we have 

Y_{f )— mm f (x) and V{f) — max f{x). 

In the sequel we will list some basic properties of lower and upper expectation 
functionals. First we have that 

E{f) = -E{-f)- 

Al the properties of lower expectation functionals are therefore easily derived 
from the corresponding properties of the upper expectation functionals, which 
are the following. Let E be an lEF, /, /i, /2 arbitrary gambles, A a non-negative 
real constant, /i an arbitrary real constant, and 1a the characteristic function 
of a set ^ C A". Then: 

(i) mm.j:(=x fix) < E{f) < mayL^^x f{x) (boundedness); 

(ii) Eih + h) < E{h)+E{h) (subadditivity); 

(iii) E{Xf) = \E{f) (non-negative homogeneity); 

(iv) E{f + ^Ix) = E{f) + ^ (constant additivity); 

(v) if /i < /2 then E{h) < E{f2) (monotonicity). 

Conversely, every functional satisfying conditions (i)-(iii) above is an upper ex- 
pectation functional for some lEF. 

We will often need the following simple result. 

Lemma 1. Let P he an expectation functional, f a gamble and a > a constant. 
Then 

(i) tff>OthenP{f)>aPil{f>a}); 
(ii) P{f) = f if and only i/P(l{^^j}) = I, 

where f = maxx^zx f{x). The above inequalities are also valid for E_ and E, 
where E is an lEF. 

Proof. To see (i) note that aly>a} < f and aP{l^f>a}) = ^(al{/>a}) < P{f) 
because of monotonicity of P. (ii) is obvious. D D D 



^This expression is a consequence of the fact that a more committal imprecise expectation 
functional is 'less imprecise', which in Walley's theory commits to accepting more gambles. 



We continue with the description of imprecise Markov chains. The imprecise 
probabihty distribution corresponding to the random variable Xn is described 
by an lEF En- Given the distribution of X„, the distribution of Xn+i (and 
consequently of the variables corresponding to later steps) can be found via 
transition probabilities: 

P{Xn+i - -iXn - x). (1) 

When the above transition probability is imprecise, it will be denoted with an 
lEF T(-\x). Given a gamble / the conditional lEF r(-|x) maps it to the set 
T{f\x). With a fixed gamble the latter expression is an interval mapping on 
the set of states X, denoted by Tf. Every Tf{x) =- r(/|x) denotes a closed 
interval \TJ {x) ,T j (x)\ = [T(/[x),r(/|a;)]. Hence, T/ is the set of gambles 
h satisfying TJ[x) < h{x) < Tf{x). The so defined operator T is called an 
imprecise transition operator (ITO). 

An ITO is a closed convex set of (precise) transition operators; however, not 
every such set is an ITO. (We will denote precise transition operators with small 
letters, usually t.) It has to be additionally required that a closed convex set 
of transition operators has separately specified rows to be an ITO, which means 
that if the transition operators t and t' belong to T then for every x E X the 
operator t" such that 

■'^" \t'f{y) iiy^x 

also belongs to T. It is easy to verify that separately specified rows of an ITO 
ensure that there is some t € T for every h € Tf so that h = tf, and in 
particular, that TJ and Tf belong to Tf. 

In order to allow calculations with imprecise transition operators, we must 
extend the definition of Tf from gambles to intervals of gambles of the form 

[lJ]:^{h:l<h<J}. 

We define 

r[/,7] = [T/,T7], 

which clearly contains exactly all the gambles of the form tf where t G T and 

Let an imprecise Markov chain be given in terms of an imprecise initial 
distribution in the form of lEF Eq, and an ITO T. The imprecise distribution 
at time n, given in the form of lEF En, is calculated with 

EM)=Eo{Ty), (2) 

where T"/ is given recursively with T"/ = / and T^+^ f — T\T^ f] for every 
fc > 0. 

The following holds. 

Proposition 1. Let lEFs E and F be given. Then 

{aE + I3F)T ^ aET + jiFT 
if Oi,l3 > and a + P — 1. 



Proof. Take any gamble /, and calculate 

{aE + fiF){Tf) = {aE + PF)i[Tf,Tf]) 

= [{aE + m{Tf), {aE + /3F)(T/)] 
= [aE{T!) + mW, aE{Tf) + (iF{Tf)] 
= a[E{Tf),E{Tf)] + fi[F{U),F{Tf)] 
= {aET + l3FT){f). 

n □ 

Much of the attention in this paper is put to invariant imprecise expectation 
functionals, which generalise the concept of invariant distributions. Let T be an 
ITO. An lEF E is called T-invariant if ET = E. Clearly, if E is T-invariant, 
then it is also T"-invariant, for every n € N . 

It has been shown previously in Skulil . |2009| that there always exists the 



least committal T-invariant lEF for every ITO T, and under certain assumptions 
also the most committal one. We have the following proposition. 

Proposition 2. Let an ITO T be given and an lEF Eq be such that either 

EqT C Eq 

or 

EoTDEo. 

Then the sequence {£'„}„£pf, where En+i = EnT, is monotone and the limit 
Eoc = linirn-oo E„ exists and is a T-invariant lEF. 

It is an immediate consequence of this proposition that taking Eq to be the 
vacuous lEF on the set of states A", the limit lEF is the unique least committal 
T-invariant lEF, but in general there is no unique most committal T-invariant 
lEF. 

Definition 1. Let E be an lEF. Then we define the support of E with 

supp(^) = {a;::B(l{,})>0}. 

The following proposition is straightforward. 

Proposition 3. For every lEF E and every gamble f G JC{X) we have that 

Eif)=E{f-lc), 
whenever supp {E) C C. 

3 Accessibility relations and convergence 

In the theory of Markov chains it is an important question which states are pos- 
sible to be visited at following times when a state has been observed at given 
time. In the case of precise Markov chains there is a clear meaning of the ex- 
pression 'possible', which states that the probability is strictly positive. While 



in the case of imprecise Markov chains the situation is more complex. The prob- 
abihties are now in the form of intervals, and these intervals can have positive 
upper bound but zero lower bound. This suggests that there may be different 
notions of accessibility relations, reflecting different notions o f 'possibility'. 



We first describe the notion of accessibility introduced bv lde Cooman et al. 



2009J, based on upper probability. This accessibility notion is based on pro- 



claiming an event possible if it has strictly positive upper probability. Note that 
the probability (interval) of the chain being in a state y at time n given that it 
is in the state x at time is obtained using formula ^ as T"l{y}(x). 

3.1 Weak accessibility relation and unique convergence 

The accessibility relation based on the upper probabilities is defined as follows. 
A state y is accessible from x in n steps if T l|j,i,(a;) > 0. We will then write 
X -^ y. When x -^ y for some n > 0, we write x -^ y. If both x -^ y and y -^ x 
then we say that the states communicate and write x «^^ y. The relation -^ is a 
preorder, i.e. is reflexive and transitive, and *~^ is an equivalence relation that 
partitions the set of states X into communication classes. Every two elements 
of a communication class are then accessible from one another. 

Definition 2. A set C C A" of states is absorbing if a; e C and x -^ y implies 
yeC. 

Proposition 4. Let C be absorbing. Then 

VcT C Vc (3) 

where Vc is the vacuous lEF on C. 

Proof. Clearly we have that supp {T{-\x)) C C for every x G C, and therefore, 
by Propositions Tf{x) = T{f\x) = T{flc\x) < Vc{f). Hence, Vc{Tf) < 
V"c(/), which implies ([3]). D D 

The following corollary follows immediately from Propositions [2] and E) 

Corollary 1. Let C be absorbing. Then there exists the unique least committal 
T -invariant lEF E such that supp {E) C C . 

Proposition 5. Let E be a T -invariant lEF. Then supp {E) is absorbing. 

Proof. Let x G supp (E) and x •^ y. By definition then there is some r > 
such that T l^yy{x) > 0. T-invariance of -E implies 

E{i[y}) = EiTi^^y}) > ;B(i{,})ri{,}(x) > 0. 

Hence y € supp {E) as well. D □ 

Take two communication classes C and T>. If for some states a; G C and 
y & T>,x leads to y, and hence every element in C leads to every element in D, 
then we say that the class C leads to T>. If a class T> only leads to itself, then 
we call it a maximal class. When there is a unique maximal class it is called 
the top class. A communication class is called regular if for every pair of its 
elements x and y there is some r such that x -^ y for every n > r. If the top 



class is regular, then the accessibility relation is said to be top class regular. An 
imprecise Markov chain is called regularly absorbing if it is top class regular and 
if there is some n G N such that 

r"lfl(y) > for every y e X\R, (4) 

where R is the top class. In other words, the lower probability that the chain will 
eventually move from any class outside the top class to the top class is positive. 
This practically means that no matter what is the initial distribution of the 
probability mass, it gets transferred to the top class, making it an absorbing 
class. So in the long run all probability mass is moved into the top class. 

The following convergence theorem is one of the most general results on 
unique convergence for imprecise Markov chains. 

Theorem 1 ( \de Cooman. Hermans, and Quaeehebeuii . l2009J . Theorem 5.1). 
Consider a stationary imprecise Markov chain with finite set X that is regularly 
absorbing. Then for every initial upper expectation Eq the upper expectation 
En = EqT for the state at time n converges point-wise to the same upper 
expectation i?oo-' 

lim E„{f) = lim EoiT'^f) -: E.^{f) for every f £ CiX). 

Moreover, the limit expectation E^o is the only T -invariant upper expectation 
onC{X). 

The main objective of this paper is to explore the situations where the as- 
sumptions of the above theorem are not satisfied. Let us first give a motivating 
example with a top class regular imprecise Markov chain that is not regularly 
absorbing, i.e. does not satisfy (JH). 

Example 1. We consider an imprecise Markov chain with the set of states X 
partitioned in two subsets Xi and X2 and ITO T that can be represented in the 
block matrix form: 

"P 0" 

R Q 



T = 



where the blocks correspond to Xi and X2 respectively, so that P is a regular 
imprecise operator, P > and R — Q. Clearly, the corresponding chain is top 
class regular, where Xi is its top class; however, it is not regularly absorbing 
because of P = 0. 

Let us first show that indeed the chain is not uniquely convergent. In fact we 
will construct invariant lEFs in three different ways, two of them being clearly 
different, while the equality between the second and the third one is not obvious 
and will turn out as a consequence of Theorem [51 

The first T-invariant lEF is the one where the probability mass is concen- 
trated in Xi, which is an absorbing set. By Theorem [T] starting with any lEF 
of the block form {El,Q) the lEFs {E^.Q) = (pi,0)r", converge uniquely to a 
T-invariant E^o = (P^,0). Where P'^ is the unique P-invariant IFF. 

Now take V to be the vacuous lEF on X. Since P = implies Q = 1, this 
implies that V{T Ix-i) = 1, as well as V{T Ix^) = 1 because of P = 1. But on 
the other hand the sequence {VT^} converges to some Poo, which must then 
satisfy both F^ilx^) = 1 and Pooll^'a) = 1- 



Yet another way to find a T-invariant lEF is the following. Let F' ~ (0, V2) 
where V2 is the vacuous lEF on X2 ■ Consider an ITO of the form 



Ti 



P 

Q* 



C T. 



For simplicity we may assume that Q* is regular, which implies that X2 is a 
communication class. The sequence F^ = F'Ti clearly satisfies the conditions 
of the Proposition [2] and therefore there exists the limit F^ — lim„_j.oo F!^ — 
{Q^F"^). Assuming regularity of Q* it is then the unique Ti-invariant IFF of 
this form. But then F'^T 3 F'^^ which by Proposition [2] again implies that the 
limit -Foo = liinn^oo F'^T"^ exists and is T-invariant. Moreover, i? > implies 

that i^oo(lA'i) — Foo{^X2) = 1- Thus the values of Foo and F^o coincide on the 
characteristic functions of both communication classes in X . It is less obvious 
though that this implies that they must be equal, as follows from our first main 
result in Theorem [2l 

Although Foo and Foo are two extremal invariant IFFs, many more can be 
constructed simply by taking their convex combinations, as easily follows from 
Proposition [TJ It is still an open question, however, whether every non-extremal 
invariant IFF can be constructed as a convex combination of the extremal ones. 

3.2 Strong accessibility relation 

The accessibility relation •w denotes the possibility that one element is accessible 
from another. Although, it might be that the lower probability of an x leading 
to a y is zero and the upper one is positive, which clearly can have essentially 
different implications than the case where both lower and upper probability are 
positive. Fxample [T] describes such a case. The weak accessibility relation ~~> is 
therefore not enough to describe all the relevant properties of the behaviour of 
an imprecise Markov chain. 

For this reason we will define another accessibility relation between sets of 
states as follows. 

Definition 3. Let A and B be arbitrary sets of states. We say that A strongly 
leads to B in n steps if T 1b{x) — 1 for every x G A. We will then write 
A ^ B. If there is some n > such that A ^ B then we will say that A 
strongly leads to B and write A ^ B. 

The strong accessibility relation denotes the case where the chain being in 
any state belonging to A will move to a set of states B with upper probability 
1. It should be noted that this does not mean that the chain will move from 
Aio B with certainty, but only that one of possible scenarios, described with 
the transition probabilities, is also such a certain move. And here is the crucial 
difference between the precise and imprecise case. While in the precise case a 
set cannot strongly lead into two disjoint sets, this is perfectly possible in the 
imprecise case. This fact implies that the interdependencies between classes of 
states in the imprecise case can be more complex than in the precise case. 

Note also that the case where A ^ B is different from the case where 
{x} — > B for every x £ A, although implies it. Consider the following example. 



1 1 1h 

Example 2. Let {x} -^ {y} and {y} — > {a;}. Then {x} -^ {x} for every 
fc G N and {y} -4^ {x} for every fc £ N. However, there is no n G N such that 
{x,2/} — !> {x}, whence {x,y} 7^ {x}. 

To help us analyse the properties of the strong accessibility relation — > we 
will define the following two functions. Let E be an lEF. Define 

1 otherwise 

for every A C X. 

Further let T be an ITO and define 

1 otherwise. 
The following proposition follows immediately from the definitions. 

Proposition 6. Let E he an lEF and A, B, C, D, Ai for i ~ 1, . . . ,n, sets of 
states. 

(i) Ot{A,B) = 1 if and only if ^pT{-\x){B) = 1 for every x d A. 

(a) IfACB then iPe{A) < iPe{B). 

(Hi) IfCCAandBCD then GxiA, B) < Qt{C, D). 



(iv) min Qt{A,,B) = Qt[\\A,,B 

i—l,...,n \ ^-^ 



We define the following operations: 



and 



(V-B * Bt)(B) == max i^ E iA)QTi A, B) 

A<ZX 



(Qt * Qt'){A, B) = max Qt{A, C)Qt' (C, B). 



The latter expression, when sequentially applied to the same function, allows 
defining its n-th power with 



n — times 

Proposition 7. Let E be an lEF, T and S ITOs and n a positive integer. Then 

(i) ipET = iPe*'S>t; 
(a) Ots = 6t * O5; 
(m) Ot" == e?^; 

(iv) ^pET" ^ipE* ©t/ 



10 



(^) '0[iim BT"! ^ 1™ ipE * ©T */ ^^^ Umits exist. Particularly, if ET C E, 
t/ien we /laue t/ie equality i/'fiim £T"1 = 1™ V'B * ©t- 

Proof, (i): Let us first show that ■0b * ©t < V'bt- Let (■(/'b * Qt){B) = 1 for 
some B. By definition, this impiies existence of some A such that V'-e(^) = 1 
and ©t(^, S) = 1. In other words E{1a) — 1 and Tlsix) — 1 for every x £ A. 
Hence, by Lemma [D (i), ET{Ib) = E{T1b) > 1 • ^(1^) = 1, which means, by 
definition, that iPet{B) = 1- 

Now suppose that iPet{B) = 1 for some B. By definition then ET{1b) = 
E{T\b) — 1, which by Lemma [I](ii) is only possible if E{lfj:^ ^-^i) ~ 1. 
Then we denote A = {x: T\b{x) = 1}, which then satisfies ipsiA) — 1 and 
Qt{A, B) = 1, whence {ipE * 0t)(5) = 1, and this proves that ipE * ©t > V'-BT- 

(n): By Proposition |n](i), <dTs{A,B) = 1 is equivalent to i^TS(-\x){B) = 
i^T(-\x)s{B) = 1 for every x S A. By (i), this is equivalent to (^t(|x)*0s)(^) = 
1 for every x £ A. Therefore, for every x € A we have a set Cx such that 
V'T(.|x)(Cx) = 1 and es(a,-B) = 1. Now let C = UxgA ^^ and, by Propo- 
sition |n](ii) and (iv), we still have that '0t(|x)(C') = 1 for every x £ A and 
Qs{C,B) = 1. Hence, eT{A,C) = 1 and ©s(C,S) = 1, which implies 

(©T*©s)(AB) = l- 

Conversely, if we assume that {Qt*Qs){A, B) = 1, this implies the existence 
of some C such that eT{A,C) = 1 and es{C,B) = 1. By Proposition |6](i), 
the first equality is equivalent to ipT(-\x){C^) — 1 ioi every x £ yl. Therefore 
ipT(-\x)s{B) = 1 for every x € A; which, as follows from above, is equivalent to 

©TS(A,B) = 1. 

(iii) is an immediate consequence of (ii), and (iv) follows from (i) and (iii). 

(v) Let lim„_s.oo('0B * 0t)(^) — 1- Then there must exist some no such that 
{tpE * ©y)(A) = 1 for every n > uq and this is equivalent to ipET^iA) = 1. 
Thus ET {A) = 1 for every n > no, which implies that lim„_).oo ET (A) = 1 

or '0[lim„^oo BT"](^) = 1- 

If -ET_g E then ET" C ET''^ holds for every n. Now_suppose that 
lim„^oo£^T (A) = 1. Then the same must hold for every n: ET {A) = 1. 
Hence V'b * ©t(^) = ^ f*^^ every n and so is the limit lim„^oo V'b * ©t(^) = 
1. D D 

Let us give an example where we have strict inequality in Proposition [7] (v). 

Example 3. Take the transition matrix 



T: 



1 
0.5 0.5 



for a Markov chain with the set of states X = {xi,a;2}. Clearly, starting with 
the initial probability distribution, say qo = (0.5, 0.5), we have ipsiA) = for all 
strict subsets A C X. The same holds for every distribution g„ = q^T", but the 
limit distribution is (1,0). Hence, we have '0g^(A) = for every strict subset of 
X, and the same for the limit lim,n.oo V'g„(^)- However, ip[\iYa„^^ gnld-^i}) = 1- 

Corollary 2. Let A and B he sets of states such that A^^ B. Then a sequence 
of sets 

A^Ao,Ai,...,An = B (5) 



11 



exists such that 

Ai -^ Ai^i,for every i — 0, . . . ,n ^ 1. (6) 

Proof. The relation A —^ B is equivalent to Qt"{A, B) = 1, and by Propo- 
sition [7](iii) , this is again equivalent to Q^{A,B) — 1. The existence of the 
sequence is now an immediate consequence of the definitions. D D 

3.3 Permanent classes 

Definition 4. A set A of states is called a permanent class if for every N ^ N 
'du n > N and a set B exist such that B —^ A. Any set of states that is not 
permanent will be called an impermanent class. 

In the case of precise Markov chains only absorbing classes are permanent, 
while the imprecise case allows much richer structure. 

Example 4. Consider the transition operator from Example [T] Clearly, the 
disjoint subsets of states Xi and X2 are both permanent classes, despite the fact 
that X2 'possibly' leads to Xi. 

The following proposition is immediate. 

Proposition 8. Every subset of states A such that A ^ A is a permanent 
class. 

Definition 5. A permanent class B is minimal ii B ^ B and contains no 
proper subset B' C B such that B' ^ B' . 

We now examine some properties of permanent classes. First we give some 
equivalent definitions. 

Proposition 9. Let a set of states A be given. The following propositions are 
equivalent: 

(i) A is a permanent class. 

(a) For every N Cz N an n > N exists and a sequence of sets Aq, Ai, . . . , An = 
A such that Ai — > A^+i for every i = 0. . . . , n — 1. 

(Hi) For every n G N a sequence of sets with the above properties exists. 

(iv) A permanent class B exists such that B ^)- A. 

(v) A minimal permanent class B exists such that B ^f A. 

Proof. The equivalence between (i), (ii) and (iii) clearly follows by Corollary [31 
Further, (iv) and (v) clearly imply (iii). 

Now let A be a permanent class. Then for every n we have a sequence 
A„.Oi ■ • ■ ; An^n-i such that An.i — >■ An^i+i for every n and i < n and A„.„_i ^■ 
A. Because of finiteness, there exists a set B such that B = yl„^„_i for an 
infinite number of indices n, and this implies that B satisfies (ii) and is therefore 
a permanent class. Thus, (i) implies (iv) as well. 

To see that (iv) implies (v), let A be a permanent class and Bi -^ B2 -^ 
■ ■ ■ ^ Bj. = A a maximal chain such that B^+i ■/> Bk for every fc = l,...,r — 1. 



12 



By (iv) there exists a permanent class C such that C — > Bi. But then, by 
maximahty of the chain, we must have that also Bi — )> C, and consequently, by 
transitivity, Bi -^ Bi. If there is a strict subset B' C Bi that is a permanent 
class then we take it to be a minimal one. We have then B' ^ Bi, by construc- 
tion. But Bi — >■ -B', by maximality of the chain. Therefore, by transitivity, we 
again have that B' — > B' which is now clearly a minimal permanent class, and 
such that B' ^ A. D D 

Proposition 10. Let B be a minimal permanent class. Then it is a communi- 
cation class. Moreover, there is some r G N such that 

T l{j^j(a;) > for every x,y ^ X (7) 

and 

T 1b{x) = 1 for every x G B. (8) 

This means that for some r G N the chain with the ITO T"^ restricted to B is 
regular. 

Proof. Let us first show that B is a communication class. We have that B -^ B, 
say B ^ B ioi some n gN. Take some x € B and denote B^ — {y & B : x ■/-* y} 
and further let B^ = {x' e B: x' ■/* B^}. We show that B^ — > B^. Suppose 
contrary that there is some x' e B^ such that T Ib^{x') < 1. By subadditivity 
of T"(-|x') we have that T'Ib\bA^') > T"'iBix'f-T'lBAx') > 0. Hence, 
there must be some y G B\Bx such that T l^yy(x') > 0, or equivalently, x' --^ y. 
But since y ^ i?x, V ^^ B^, which together implies that x' -^ B^, which is in 
contradiction with x' G B^. 

It is therefore confirmed that B^ -^ B^, which contradicts minimality of B, 
unless Bx = 0. Together this proves that i? is a communication class. 

Let us now show that some r G N exists so that equations ([7]) and ([5]) are 
satisfied. Let x E B and denote with r^ the greatest common divisor of all r 
such that X -^ X. Now denote Cx ^ {y (z B : 3k, x -^ y}. 

Since T ^ 1_b\Cx iv) ~ ^ ^'^^ every y € Cx, which implies that T ^ 1b^ {y) = 1 

for every y (z C'x- Thus, Cx — ? Cx and therefore, Cx -^ Cx- Since i? is a minimal 
permanent class and Cx a non-empty subset, it must therefore hold that Cx = B 
for every x E B. 

We now show that there is some r G N such that T l^y^{x) > for every 
x,y G B. It follows from the definition of rx and the elementary properties 
of accessibility relations that there is some L G N such that x ^ x for every 
I > L, and for every y E B there exists some ly such that x -^ y. Denote 
Mx — iaaxy,=B{L + ly). Then x -^ y for every y € B and every m > Mx. Now 

denote R — YixeB ^^ ^'^'^ ^ ~ niaxj^gs Mx- Clearly then x -^ y for every x 
and y d B, whenever m > M . 

Because _B is a minimal permanent class, we have that B ^ B. If s = 1 
then (|5]) is satisfied for every r G N and therefore also for r — mR. While in the 
case where s > 1 we can replace T with T* and reason as above to show that 
(^rpsynR _ rpsrriR ga^igggg both properties required, since, as can easily be noted, 
B is still a minimal permanent class when T is replaced with T*. D D 
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Note that the above proposition does not claim that a minimal permanent 
classes would be whole communication classes, but merely that every minimal 
permanent class lies in a single communication class. 

Definition 6. Let i? be a minimal permanent class, such that B ^ B. Then 
we define 

T^j := {f e T^ tlsix) = 1 for every x e B}. (9) 

Proposition 11. Let B be a minimal permanent class, such that B -^ B. Then 
B is regular and absorbing with respect to the weak accessibility relation induced 
by {TgY for some k eN. 

Proof. Let -^ and — > now denote the weak and the strong accessibility relation 
induced by Tg. By construction we have that B ^- B and since T]j is a subset 
of T"^ this relation cannot hold for any proper subset of B. Therefore i? is a 
minimal permanent class with respect to T^ as well. 

By Proposition [TO] it is then a communication class and, moreover, for some 
A: £ N the power {T'^Y restricted to B is regular. By construction, B is also 
absorbing with respect to Tg and therefore, with respect to {Tg)^ as well. D 

D 

4 Unique convergence for extremal lEFs 

In this section we use the results prepared in previous sections to state our 
main results on convergence of imprecise Markov chains. We will explore the 
convergence of the class of so called extremal lEFs, which are those which induce 
the upper probabilities of minimal permanent classes being either or 1. In the 
next subsection we show that when the sequence {ET^^} consists of extremal 
lEFs this sequence converges if the sequence E{T'^1b) converges (either to or 
1) for every minimal permanent class B, and the limit is uniquely determined 
by the limits lim„_i.oo E{T^^1b) for all minimal permanent classes B. 

Moreover, in Subsection 14.21 we show that it is sufhcient for the unique con- 
vergence that the limits lim„_j.oc E(T^^Ib) are zero-one valued, rather than all 
terms of the sequences. 

4.1 Uniqueness of invariant imprecise expectation func- 
tionals 

The importance of permanent classes is illustrated with the following proposi- 
tion. 

Proposition 12. Let M be the unique least committal T -invariant IFF. Then 
ipM (A) ^ I if and only if A is a permanent class. 

Proof. The unique least committal lEF M is equal to lim„_^oo VT", where V 
is the vacuous lEF on X. Since VT C V, Proposition [7](v) gives that 

V'M = V'fiim,,^^ yT"i = lim V'y * ©t- 

n— ^oo 

If i^MiA) = 1, this implies that for every n G N we have that ipv * Q^{A) = 1, 
or equivalently, there exists some B such that ipv{B) = 1 and B ^ A. Hence, 
A is a permanent class. 
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Moreover, if A is a permanent class, then for every large enough n £ N there 
exists some B such that B —^ A, and since ipviB) = 1 for every B C X, then 
tpv * 0t(^) = 1- As this holds for every large enough integer n, so it does for 
the limit, and implies that iPm{A) = 1. D D 

Definition 7. Let E be an lEF and / a gamble. Then we define essential 
maximum of / given E with 

ess max b/ ~ max{a: E^{l{f>a}) > 0}, 

and 

rUE = min{^(lA): A C X,E{1a) > 0}. 

Proposition 13. Let E and F be lEFs and f a gamble. The following propo- 
sitions hold: 

(i) E{f) < ess max b/. 

(a) /// > then E_{f) > ttie ■ ess max s/. 

(Hi) IfijJE = ^_F7 then ess max j?/ = ess max b/. 

(iv) Let {En} be a sequence of lEFs such that En+i C En- Then lim„_i.oo '^e^ 
exists and there is an N € N such that {mE„}n>N is a non- decreasing 
sequence. 

Proof. Let in this proof / denote ess max s/. 

To see (i), take some P ^ E such that P{^tfyf\) — 0. This means that 
P(l{j.}) = for every x & X such that f{x) > /. Then 

Eif) < pif) = E ^(^)/(^) = E ^(^)/(^) < /■ 

2:6^ x: f{x)<f 

By Lemma [1] (i) and (i) of this proposition, we obtain 
Mf) > /£(l{;>/j) > fruE, 

which proves (ii). 

To see (iii) notice that tpE = ijjp if and only if ^(l{/>a}) > whenever 
F(l|/>„}) > 0. 

(iv): To see this note that the set {E_^{1a). n G N} is a non-decreasing 
sequence for every A C X, either constantly equal to 0, or contains a minimal 
non-zero element. Because of the finite number of subsets of X we have at most 
a finite number of positive minima, whose minimum exists and is positive as 
well. 

Clearly, there is also an iV G N such that for every A C X either ^„(lyi) > 
for every n > N or S„(l^) = for every n > N. Therefore, {mE„}n>N is a 
non-decreasing sequence. D D 

Corollary 3. Let E and F be lEFs such that tpE — i'F- Then 

E{f)>mE-F{f) (10) 

for every gamble / > 0. 
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Proof. By Proposition fTSTii) we have that E_{f) > itie ■ ess max s/, and by (i) 
of the same proposition we have that F_{f) < ess max i?/ — ess max s/. This 
together imphes dTU]). D D 

Corollary 4. Let E and F he lEFs such that ipE = ipF, supp (F) C supp (E), 
and Ve the vacuous lEF on supp (E) . Then 

ECmEF+{l-mE)VE. 

Proof. We need to prove that 

Eif) > mEEif) + (1 - mE)VE{f) (11) 

holds for every gamble / G J0-{X) . Let / be a gamble and denote h ~ f- Igupp (^e) ■ 
Then, by Proposition [31 we have that H{f) = H{h) where H stands for E,F 
and Ve respectively. 

Denote h = T^i^xesupp {e) f{x) — Hsl/)- Then we have that h ~ h > 0. 
By Corollary 13] we have that E_(h — tk) ^ mEFjJi — h). Hence, by constant 
additivity of E_ and F, E_{h) — h> mEF_{h) — niEh holds. By replacing h with 
VEif),Mh) with E{f) and F{h) with F{f) we obtain 

Mf) - VeU) > ^EFif) - ^ieVeU), 

whence inequality (fTTj) follows. D D 

Definition 8. Let E be an lEF such that E{1b) G {0, 1} for every minimal 
permanent class B. Then we say that E is an extremal lEF. 

In the precise case the concept of an extremal expectation functional is 
rather trivial. Note that a minimal permanent class in the precise case can only 
be absorbing. Therefore having the entire probability mass once concentrated 
in such a class it will remain there forever, and additionally, it can only be 
concentrated in one such class at the time. Consequently, if the class is aperiodic, 
the probability distributions will then converge to a unique distribution invariant 
for the chain restricted to this, regular and absorbing class. 

Definition 9. Let E be an lEF. Then we define 

Se ^ {y.^x e supp(£'),x -> y} 
and Ve will from now on denote the vacuous lEF on Se- 
Clearly, the following holds. 

Proposition 14. For every lEF E, the set Se is absorbing. Moreover, the 
sequence 

is monotone: AIn+i '~= Mn, and the limit 

M = lim Mn 
exists and is T -invariant. 
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Proof. It is an immediate consequence of the definition that Se is absorbing. 
The convergence then follows from Propositions [2] and 21 D D 

Lemma 2. Let A be a permanent class such that A ^ A and S an absorbing 
set. Then AnS ^ AdS. 

Proof. Clearly, S being absorbing implies that T lc{x) = for C fl S* = 
and a; G 5. Now take some x G An S. By subadditivity of T{-\x) we have 
that 1 = T 1^(2;) < T lAnsix) + T Ia\s{x)\ whence, by T Ia\s{x) = 0, 
T''lAns(a;) = 1 follows. D D 

Corollary 5. Let B be a minimal permanent class and S an absorbing set. 
ThenBnS e {B,0}. 

Proposition 15. Let A ^ B and let E be an lEF. Then E{T"^Ib) > E{Ia)- 

Proof We have that E{T'Ib) > E{1a) ■ min^^^ r"ls(x) = E{Ia). □ □ 

Corollary 6. Let A ^ B and let E be a T-invariant lEF. Then 'E{1b) > 
E{Ia). 

Lemma 3. Let E be an lEF such that ET"' is extremal for every n > and 
the limit lim„_^oo ipET"^{B) exists for every minimal permanent class B. Then 
the limit 

tP{A) := lim ijET-^iA) 



n— >oo 



exists for every A C X and tp is uniquely determined by its restriction to minimal 
permanent classes. 

Proof. Denote En = ET". Let us fist show that for every minimal permanent 
class B lim„_j.oo V'B„ (-S) = 1 if and only if B C Se- To see this, take some 
minimal permanent class B C Se and note that £^^(1^) > must hold by 
definition of Se for some fc > 0. Extremality of E^ implies that Ekils) = 
1, and the fact that B -^ B, say B ^f B^ hy Proposition [T51 implies that 
Ek+mri^B) = 1, for every ?7i e N, whence, by the assumed convergence of the 
sequence {'0_e„(5)}, we must have that, for some A'' e N, ipE^iB) = 1 for every 
n>N. 

On the other hand, if i? ^ Se, then by Corollary[5l BDSe = 0, and therefore 
clearly, Enils) = for every n > 0. 

Now let A C X. Then either ipEni^) = for all sufficiently large n or for 
every N G N there exists an n > A^ such that tpE„{A) = 1. The former case 
implies convergence, so let us show that the latter case does as well. Suppose 
that ipE„i^) = 1- By Proposition[7](iv) we have that iI^eA^) = (V'b*0t)(^) = 
1, which implies the existence of a sequence Aq ^ Ai -^ . . . -^ An — A, such 
that ■0b(^o) = 1- If ?T- is sufficiently large, then at least two members of the 
sequence are equal, say A^ ^ Ai = A' . Hence, we have A' -^ A' ^ A. By 
Lemma [5] we have that B' — A' r\ Se also satisfies B' -^ B', and moreover 
_B' — s> A' — > A. However, B' either is a minimal permanent class or contains 
one. Let B denote such a minimal permanent class, which by construction 
lies within Se, and therefore, by the assumptions, an A^ G N exists such that 
ipEn (-S) = 1 for every n > N. We have that B —i' A, say B ^ A. Moreover, by 
Proposition [TS] we have that ipE^+ni^) — '^e„{B) = 1 for every n> N. Hence, 
iPb^ (A) = 1 for every k>r + N. D D 
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Theorem 2. Let E he an lEF such that En = ET"^ is extremal for every 
71 > and the limit lim„^oo ipE„ {B) exists for every minimal permanent class 
B. Then the limit 

Eoc ■= lim En 

exists. Moreover, lini„_).oo En — lini„_^oo -Fn */ cind only if Se — Sp- 

Proof. By Lenima|3]and the assumptions of the theorem, the hmit ip = hm„_i.oo 'ipE„ 
exists, and since ipE„ are discrete valued, there is some iVi £ N such that 
V'-En = "0 ioT every n > Ni. 

It has been shown in the proof of Lemma [3] that ip{B) = 1 for exactly those 
minimal permanent classes B that are contained in Se- 

Let Ve be the vacuous lEF on Se and denote M„ — VeT"- and M — 
lim„_^.oo Mn, which exists by Proposition 1141 Similar arguments as above show 
that for every minimal permanent class B we have that iPmIB) = 1 if and only if 
B C Se, whence ipM — "0- Moreover, by Proposition [7](v), ipM — lini„^oo V'Afn, 
and therefore, there is some N2 ^N such that '0m„ ~ i'M ~ "0 ^oi every n > N2. 

Because of £' C Ve, we also have that supp (-E„) C supp(Af„) for every 
n G N. Now let N = max{iVi,A^2} and denote m — mM^ (c.f. Definition [T]) . 
By Corollary m we then have that 

Mn C tuEn + (1 - m)VE. 

Now take any gamble f ^ C and any n > 0. By Proposition [1] we obtain 

MN{T'f) < mEN{Tf) + (1 - m)Fi5(T7). 

Letting n -^ 00 va the above equation we obtain 

M{f) < mhminiEnif) + (1 - ™)M(/), 

n— J-oo 

implying that M{f) < liminf„^oo ^«(/)- But since, clearly, M„(/) > En{f) 
for every n S N, this is only possible if the limit 

Eoo(/) = lim Enif) 

exists and is equal to M{f). Since this holds for every gamble /, we have that 
Eoo = M . Now since M is uniquely determined by Se, the limit is the same for 
all lEFs E that have the same Se. □ □ 

4.2 Unique convergence to extremal invariant lEFs 

In the previous subsection we have shown that extremal lEFs converge to a 
unique invariant lEF that is uniquely determined with the behaviour of the 
corresponding '^pE restricted to minimal permanent classes. But often the prob- 
ability mass is not initially entirely concentrated in a minimal permanent class, 
but rather it accumulates there as time goes to infinity. It may then happen 
that in the limit the upper probabilities of all minimal permanent classes are 
either or 1, although in any finite time they can be somewhere between. In 
this subsection we show that a convergence to an extremal lEF is unique when- 
ever the upper expectations on minimal permanent classes converge to either 
or 1. Moreover the limit lEFs are still uniquely determined with the behaviour 
on the minimal permanent class structure. 
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Proposition 16. Let A be an absorbing set. Then 1a ■ T"/ = 1^ • T"[1a ■ f] 
for every n £ N. 

Proof. The fact that A is absorbing implies that T1a<^{x) — for every x € A, 
and consequently, Tf{x) = T[lAf]{x) for every x £ A. Hence 

lATf=lAT[lAf]. (12) 

But if A is absorbing for T, then it is also absorbing for every T", where n E N, 
and therefore the equation ([121 holds when T is replaced with T". D D 



Proposition 17. Let A be an absorbing set such that the weak accessibility 
relation induced by T and restricted to A is regular, and E an lEF such that 

supp {E) C A. (13) 

Then 



Ma := lim ET" 
exists and is the same for every E satisfying (|13p . 
Proof. Let us first define the following transition operator: 

[VAf x^A, 

where Va is the vacuous lEF on A. Clearly, the chain with the ITO Ta is 
regularly absorbing with the regular top class A. Therefore, by Theorem [TJ the 
sequence {ET^} converges uniquely to some Ma- Furthermore, Proposition llGI 
and the definition of Tyi imply that UTl/ = UTliU/] = 1aT"[1a/] = UT"/ 
for every gamble /. Using Proposition |3] and the above equalities we obtain 

E{T^f) = E{lAT^f) = E{lATlf) = E{Tlf), 

and since the right hand sides converge, so do the left ones, for every /. D D 

Corollary 7. Let A be a set of states and T' C T an ITO such that A is 
absorbing and regular with respect to the weak accessibility relation induced by 
T' . Then there exists a T -invariant lEF Ma such that for every lEF E with 
E{1a) — \ we have that 



\immiE{T"f)>MA{f) (14) 



for every gamble f. 



Proof. Since E{Ia) ~ 1, there exists an lEF E' C E such that supp(£") C 
A. Denote E'^ = E'T'". By Proposition [HI the hmit M'^ := lim„^oo E'„ 
exists and is the unique such limit lEF corresponding to T'. Moreover M^ = 
M'^T' C M'^T and therefore, the limit Ma = lim„^oo M^T" exists as well, by 
Proposition [21 and is T-invariant. We will show that it satisfies ((HI). 

The convergence of the sequence {E'^} to M'^ implies that for every e > 
there exists an iV e N such that E[^{f + elx) = E'„{f) +e> M'^{f) for every 
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n > N and every f £ Ci, which is a compact set of gambles. Further, we denote 
E„ = ET", and clearly E,, 2 K holds. Hence, 

EN+k{f)+e = EN+k{f + slp,)>E'^+kif + £^x) = E'N+k{.f)+e>M'AT''f). 
Letting fc ^ oo on both sides we obtain that 

liminfE„(/)+e>M^(/) 

for every e > 0, and therefore liminf„^oo -&„(/) > Af^(/) for every / € £1, 
and therefore also for every gamble / G £. D D 

Corollary 8. Let A, T' and Ma satisfy conditions as in Corollary^ and let E 
he an IFF such that 

lim E(T'\a) = 1. 

Then 

limmiE{Tf)>MA{f) (15) 

for every gamble f. 

Proof. Let us first show that, for some constant a, F(Ia) > a implies that 

liminf F(T"/) > oMAif) (16) 

for every gamble /. By Corollary [7] we have that 

liminfP(T"/)>MA(/) (17) 

n— >oo 

for every (imprecise) expectation functional P such that P{1a) — 1- Now 
suppose that F{1a) > a. Then there exists some P € F such that P{1a) > ct, 
and by definition we have that P{T /) < F{T f) for every n G N. Moreover, 
we have that 

P{Tf) - P{lA)P{Tf\A) + P{lA^)P{Tf\A^), 

where P{-\A) is the conditional expectation functional. Hence, 

P(T"/) > «^(T"/|A) = aP(-|A)(T"/). 

However, by (HT]), liminf„^oo -P(-|^)(r"/) >MA{f), whence 

liminf F(T"/) > liminf P(T"/) > oMaU)- 

and this proves ([T5|. Now, for every e > there is some n e N such that 
E{T 1a)>1 — £. Hence, for every gamble /, 

liminf ;E(T"t''7) > (1 - eWAiT^f) = (1 - e)MA{f). 

Since the above holds for every e > 0, inequality ([T5]) follows. D D 
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Corollary 9. Let B be a minimal permanent class. Then a T -invariant lEF 
Mb exists such that for any lEF E with 

lim :B(T"1b) = 1. (18) 

the inequality 

\iminiE{Tf)>MB{f) (19) 

n— )-oo 

holds for every gamble f . 

Proof. By Proposition [11] B is absorbing and regular with respect to {T"^)^ C 
T''^. Therefore, by CoroUary[51 lEF Mb exists such that 

hminfE(T"'7)>MB(/) 

for every lEF E satisfying ((T5)) . and therefore also for ET"^, where m is arbitrary 
integer. Thus we have 

liminf;B(T™+"''7) - liminf M^(T"''7) > Msif)- 

It remains to prove that AIb is T-invariant. By Corollary[71 it is T^'^'-invariant. 
Therefore MbT = AfsT"'^'''+i. Replacing E with Mb in ((HI), and using the 
above identity imply that MB{Tf) > MB{f) for every gamble, which implies 
the inclusions Mb C MbT C • • • C MbT^'' = Mb. But then all the inclusions 
must in fact be equalities, particularly, Mb = MbT must hold. D D 

Theorem 3. Let E be an lEF on X such that the limit 

lim E{t"Ib)^:E^{Ib) 

n— ^cx3 

exists for every minimal permanent class B, and i?oo(ls) G {0,1}- Then the 
limit 

lim E{Tf)=:EUf) 

n— >oo 

exists for every gamble f. Moreover, E^ is the unique T-invariant lEF such 
that EocI^b) = 1 for every minimal permanent class B C Se. 

Proof. Let i? be a minimal permanent class such that 

lim ^(T"1b) = 1. (20) 

Then by Corollary [S] we have that liminf„^oo £-(2^ /) > MB{f)- Let us show 
that this holds for every minimal permanent class B C Se- Suppose contrary 
that there is some B C Se such that liuin-^oo E{T Ib) ^ 1- Then, by the 
assumption, lining oo E(T Ib) — 0. But, by definition of Se, there must be 
some m > such that E{T Ib) > 0, which by Proposition [T51 implies that 
E{T Ib) > E{T Ib), and therefore the limit as n ^> oo cannot be 0. 
This contradiction confirms that ([20)) holds for every minimal permanent class 
BCSe. 

By the above, for every minimal permanent class B C Se we have that 
liminf„_^oo^(T"/) > Mi3(/)- Let us define 

MEif) = max MBif), 
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which is an lEF with Me{1b) = 1 for every B C Se- Moreover, since 
JlEiTf) = max MB{Tf) = max Mb(/) = M£(/). 

it is a T-invariant lEF, and by Theorem [21 it must therefore be the unique 
T-invariant lEF with these properties on Se- Hence, 

Me = hm y^T", 

where Ve is the vacuous lEF on Se- For the vacuous IFF we have that E C Ve, 
and therefore, for every gamble / we have that 

Uminf;E(T"/) > Mb(/) = hm V e(t'' f) > hmsupE(T"/). 

Hence, liminf„^oo^(T7) = hmsup„_^^;E(T"/) = hm„^oo^(r"/) = Me{I)- 

U U 

Let us conclude this section with an example. 

Example 5. Let <Y be a set of states partitioned into Xi, X2 and A3. Suppose 
that an ITO can be decomposed in the following matrix form; 

/Til 
T = Tai T22 

where Tulxiix) = 1 for every i = 1,2,3 and every x G Xi. Moreover, assume 
that T induces a regular weak accessibility relation on every Xi. Then Xi are 
all minimal permanent classes. If T^ = whenever i ^ j, then we can have 
that E{lxi) equals or 1 independently of each other, with the only condition 
that E{lx) = 1. Therefore we have seven different extremal T-invariant lEFs. 

However, in the case where T^ > for some i ^ j, E{lxi) clearly implies 
that Vniin^oc E{T Ix) = 1 as well, which restricts the number of possible 
distinct extremal lEFs. 

In addition to the extremal T-invariant lEFs, we can obtain the non-extremal 
ones by forming convex combinations. It is one of the challenges left to explore, 
whether every non-extremal T-invariant IFF is equal to such a convex combina- 
tion. If the answer is positive, this would allow us to characterise all T-invariant 
lEFs. 



5 Conclusions 

The first main contribution of this paper is the introduction of the so-called 
strong accessibility relati o n wh ich together with the weak relation, defined ear- 
lier by Ide Cooman et al.l 2009| , allows a detailed analysis of the behaviour of 



imprecise Markov chains. We have identified the minimal sets of states, named 
the minimal permanent classes, that correspond to essential communication 
classes in the classical theory. They are, similarly, the minimal sets of states 
where the entire probability mass can be concentrated and possibly remains 
there forever. 
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Once the mininial irreducible building blocks had been identified, we could 
describe and analyse the invariant distributions whose probability mass is en- 
tirely concentrated within the minimal permanent classes, and this is also the 
point of the crucial difference between precise and imprecise theory. While in 
the precise theory, the entire probability mass can be in one class at the time 
only, it can 'possibly' be in several classes when the 'imprecise interpretation' is 
adopted. Therefore the behaviour of an imprecise Markov chain cannot be com- 
pletely reduced to the behaviour within minimal permanent classes. Instead, we 
study the so-called extremal imprecise expectation functionals, which allow the 
probability mass to be concentrated entirely in one or more permanent classes 
at the time. 

The extremal imprecise expectation functionals seem to be the minimal ob- 
jects in the class if imprecise expectation functionals that cannot be further 
reduced to simpler constituent parts. Our main convergence result in Theo- 
rem [5] then shows that when an invariant imprecise distribution is extremal 
then it is uniquely determined with the corresponding set of possible minimal 
permanent classes. Moreover, as shown in Theorem [31 the unique convergence 
to extremal invariant distributions is ensured already with the convergence of 
the restrictions to the characteristic functions of minimal permanent classes to 
either or 1. 

Convex combinations of extremal invariant imprecise expectation functionals 
are the limits of the same combinations of the initial imprecise expectation 
functionals. Two important challenges that remain subject to further work arise. 
First is to explore whether every invariant imprecise expectation functional is 
only a convex combination of a family of extremal ones, and the related question 
is whether unique convergence when restricted to minimal permanent classes is 
enough for unique convergence of imprecise Markov chain. Another challenge is 
the analysis of periodic behaviour in imprecise Markov chains. 
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